(depends on #10217)

This change causes ls-apis to parse package-manifest.toml to figure out what commits of related repos (like Crucible, Dendrite, etc.) will actually be deployed (from the current Omicron workspace). It then uses this information to choose the correct clone of the repo to use for its analysis.

One other change I made here was to tie lldpd-client and its protocol package to the version that's deployed in package-manifest.toml. This ought to fix #10361. (A previous version of this PR updated package-manifest.toml instead, but I opted for the smaller change here.)

Background

ls-apis needs access to checked-out repos for Omicron as well as related components like Dendrite, LLDP, Crucible, Propolis, etc. It wants the versions of these repos that get deployed on real systems (based on the Omicron workspace that it's running in), since the goal is to analyze the runtime API dependencies between these components. It could create its own clones of these repos, but instead, it leverages the fact that just running cargo metadata in Omicron requires having downloaded copies of all of these repos already. How does ls-apis find these copies? It uses Cargo to locate a package that's known to be in that repo. Generally, it picks the package of a client that Omicron already from that repo, like dpd-client to find Dendrite.

But it's not quite so simple: Omicron can reference multiple versions of a given repo. More specifically: Omicron may reference dpd-client from multiple versions of Dendrite. This happens with dpd-client specifically:

$ cargo tree -e normal -i dpd-client
error: There are multiple `dpd-client` packages in your project, and the specification `dpd-client` is ambiguous.
Please re-run this command with one of the following specifications:
  git+https://github.com/oxidecomputer/dendrite?branch=main#dpd-client@0.1.0
  git+https://github.com/oxidecomputer/dendrite?rev=44a949c9bedf4fcd4d280337fa1965b4293c88d1#dpd-client@0.1.0
  git+https://github.com/oxidecomputer/dendrite?rev=cc8e02a0800034c431c8cf96b889ea638da3d194#dpd-client@0.1.0

$ cargo tree -e normal -i git+https://github.com/oxidecomputer/dendrite?branch=main#dpd-client@0.1.0
dpd-client v0.1.0 (https://github.com/oxidecomputer/dendrite?branch=main#f20f786e)
└── lldpd-common v0.1.0 (https://github.com/oxidecomputer/lldp?rev=c3305fd1a7ea7aba31f3834757a6b931e4f59fe6#c3305fd1)
    └── lldpd-client v0.1.0 (https://github.com/oxidecomputer/lldp?rev=c3305fd1a7ea7aba31f3834757a6b931e4f59fe6#c3305fd1)
        └── omicron-nexus v0.1.0 (/home/dap/omicron-review/nexus)
            └── omicron-dev v0.1.0 (/home/dap/omicron-review/dev-tools/omicron-dev)

$ cargo tree -e normal -i git+https://github.com/oxidecomputer/dendrite?rev=44a949c9bedf4fcd4d280337fa1965b4293c88d1#dpd-client@0.1.0
dpd-client v0.1.0 (https://github.com/oxidecomputer/dendrite?rev=44a949c9bedf4fcd4d280337fa1965b4293c88d1#44a949c9)
├── nexus-test-utils v0.1.0 (/home/dap/omicron-review/nexus/test-utils)
│   └── omicron-dev v0.1.0 (/home/dap/omicron-review/dev-tools/omicron-dev)
├── omicron-nexus v0.1.0 (/home/dap/omicron-review/nexus)
│   └── omicron-dev v0.1.0 (/home/dap/omicron-review/dev-tools/omicron-dev)
├── wicket-common v0.1.0 (/home/dap/omicron-review/wicket-common)
│   ├── wicket v0.1.0 (/home/dap/omicron-review/wicket)
│   │   └── wicket-dbg v0.1.0 (/home/dap/omicron-review/wicket-dbg)
│   ├── wicketd v0.1.0 (/home/dap/omicron-review/wicketd)
│   ├── wicketd-api v0.1.0 (/home/dap/omicron-review/wicketd-api)
│   │   ├── omicron-dropshot-apis v0.1.0 (/home/dap/omicron-review/dev-tools/dropshot-apis)
│   │   └── wicketd v0.1.0 (/home/dap/omicron-review/wicketd)
│   └── wicketd-client v0.1.0 (/home/dap/omicron-review/clients/wicketd-client)
│       ├── wicket v0.1.0 (/home/dap/omicron-review/wicket) (*)
│       └── wicketd v0.1.0 (/home/dap/omicron-review/wicketd)
└── wicketd v0.1.0 (/home/dap/omicron-review/wicketd)

$ cargo tree -e normal -i git+https://github.com/oxidecomputer/dendrite?rev=cc8e02a0800034c431c8cf96b889ea638da3d194#dpd-client@0.1.0
dpd-client v0.1.0 (https://github.com/oxidecomputer/dendrite?rev=cc8e02a0800034c431c8cf96b889ea638da3d194#cc8e02a0)
└── omicron-sled-agent v0.1.0 (/home/dap/omicron-review/sled-agent)
    ├── end-to-end-tests v0.1.0 (/home/dap/omicron-review/end-to-end-tests)
    └── nexus-test-utils v0.1.0 (/home/dap/omicron-review/nexus/test-utils)
        └── omicron-dev v0.1.0 (/home/dap/omicron-review/dev-tools/omicron-dev)

This is almost certainly not great. But it shouldn't cause ls-apis to break. Right now if this happens, ls-apis picks one of these arbitrarily, which can cause it to analyze the wrong version of our software and draw wrong conclusions. This is the real cause of #10214.

Again: we want ls-apis to be looking at the version of these things that gets deployed. How can it know which one it is? The authoritative version is the one in package-manifest.toml. Hence the solution here: parse that file, find the commit being used there, and choose the version of the package that corresponds to that commit.

Other notes

This is still a little cheesy in a few ways:

• to determine if it's the right commit, we do a string comparison on the "source", which is basically a git URL
• it's still using this sort of goofy heuristic (find a known package referenced by Omicron but contained in the other repo) -- just to be able to re-use Cargo's clone of the repo. It's pretty nice to not have to manage a separate set of clones (and make sure they're correct, have no local changes, etc.), but it's kind of a weird assumption to make and it would break if we ever had a repo we cared about where Omicron doesn't reference one of its packages.

but I think it's a meaningful improvement.

One other note: this will break in the future if:

• Omicron has no reference at all to a package in the other repo (the case above -- this would already have broken before)
• Omicron has no reference to a package in the other repo at the same Git commit as the one in package-manifest.toml. This would be unlikely, since we usually move all of our deps forward at the same time. But there's a notable exception today: one of the dpd-client paths above is deliberately fixed to an old version for upgrade-related reasons. This works out fine though because there's another reference to dpd-client that is the right version.
• package-manifest.toml references the same repo multiple times with different commits. Again, this seems unlikely, and the problem is deeper than it looks. That means we have two different versions of something deployed and ls-apis needs to analyze both?